Improving Cluster Management with Scalable Filesystems

Reducing the complexity of the hardware and software components of Linux cluster systems can significantly improve management infrastructure scalability. Moving parts, in particular hard drives, generate excess heat and have the highest failure rates among cluster node components. The use of diskless nodes simplifies deployment and management, improves overall system reliability, and reduces operational costs. Previous diskless node implementations have relied on a central server exporting node images using a high-level protocol such as NFS or have employed virtual disks and a block protocol such as iSCSI to remotely store the root filesystem. We present a mechanism to provide the root filesystems of diskless computation nodes using the Lustre high-performance cluster file system. In addition to eliminating the downtime caused by disk failures, this architecture allows for highly scalable I/O performance that can be free from the single point of failure of a central fileserver. We evaluate our management architecture using a small cluster of diskless computation nodes and extrapolate from our results the ability to provide the manageability, scalability, performance and reliability required by current and future cluster designs.